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Abstract 


Although widely used (and most of the times unnoticed), DNS (Domain 
Name System) is too much overlooked, in the sense that people, 
especially administrators, tend to ignore possible anomalies as long 
as applications that need name-to-address mapping continue to work. 
This document presents some tools available for domain administrators 
to detect and correct those anomalies. 


1. Introduction 


Today more than 3,800,000 computers are inter-connected in a global 
Internet [1], comprising several millions of end-users, able to reach 
any of those machines just by naming it. This facility is possible 
thanks to the world widest distributed database, the Domain Name 
System, used to provide distributed applications various services, 
the most notable one being translating names into IP addresses and 
vice-versa. This happens when you do an FTP or Telnet, when your 
gopher client follows a link to some remote server, when you click on 
a hypertext item and have to reach a server as defined by the URL, 
when you talk to someuser@some.host, when your mail has to be routed 
through a set to gateways before it reaches the final recipient, when 
you post an article to Usenet and want it propagated all over the 
world. While these may be the most visible uses of DNS, a lot more 
applications rely on this system to operate, e.g., network security, 
monitoring and accounting tools, just to mention a few. 


DNS owes much of its success to its distributed administration. Each 
component (called a zone, the same as a domain in most cases), is 
seen as an independent entity, being responsible for what happens 
inside its domain of authority, how and what information changes and 
for letting the tree grow downwards, creating new components. 
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On the other hand, many inconsistencies arise from this distributed 
nature: many administrators make mistakes in the way they configure 
their domains and when they delegate authority to sub-domains; many 
of them don’t even know how to do these things properly, letting 
problems last and propagate. Also, many problems occur due to bad 
implementations of both DNS clients and servers, especially very old 
ones, either by not following the standards or by being error prone, 
creating or allowing many of the above problems to happen. 


All these anomalies make DNS less efficient than it could be, causing 
trouble to network operations, thus affecting the overall Internet. 
This document tries to show how important it is to have DNS properly 
managed, including what is already in place to help administrators 
taking better care of their domains. 


2. DNS debugging 


To help finding problems in DNS configurations and/or implementations 
there is a set of tools developed specifically for this purpose. 
There is probably a lot of people in charge of domain administration 
having no idea of these tools (and, worse, not aware of the anomalies 
that may exist in their configurations). What follows is a 
description of some of these programs, their scope, motivations and 
availability, and is hoped to serve as an introduction to the subject 
of DNS debugging, as well as a guide to those who are looking for 
something to help them finding out how healthy their domains and 
servers are. 


Some prior knowledge from the reader is assumed, both on DNS basics 
and some other tools (e.g., dig and nslookup), which are not analyzed 
in detail here; hopefully they are well-known enough from daily 
usage. 


2.1. Host 


Host is a program used to retrieve DNS information from name servers. 
This information may be used simply to get simple things like 
address-to-name mapping, or some more advanced purposes, e.g., 
performing sanity checks on the data. It was created at Rutgers 
University, but then Eric Wassenaar from Nikhef did a major rewrite 
and still seems to be actively working on improving it. The program 
is available from ftp://ftp-.nikhef.nl/pub/network/host_YYMMDD.tar.Z 
(YYMMDD is the date of the latest release). 


By default, host just maps host names to Internet addresses, querying 
the default servers or some specific one. It is possible, though, to 
get any kind of data (resource records) by specifying different query 
types and classes and asking for verbose or debugging output, from 
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any name server. You can also control several parameters like 
recursion, retry times, timeouts, use of virtual circuits vs. 
datagrams, etc., when talking to name servers. This way you can 


simulate a resolver behavior, in order to find any problems 
associated with resolver operations (which is to say, any application 
using the resolver library). As a query program it may be as 
powerful as others like nslookup or dig. 


As a debugger, host analyzes some set of the DNS space (e.g., an 
entire zone) and produces reports with the results of its operation. 
To do this, host first performs a zone transfer, which may be 
recursive, getting information from a zone and all its sub-zones. 
This data is then analyzed as requested by the arguments given on the 
command line. Note that zone transfers are done by contacting 
authoritative name servers for that zone, so it must be possible to 
make this kind of request from such servers: some of them refuse zone 
transfers (except from secondaries) to avoid congestion. 


With host you may look for anomalies like those concerning authority 
(e.g., lame delegations, described below) or some more exotic cases 
like extrazone hosts (a host of the form host.some.dom.ain, where 
some.dom.ain is not a delegated zone of dom.ain). These errors are 
produced upon explicit request on the command line, but you may get a 
variety of other error messages as a result of host’s operations, 
something like secondary effects. These may be mere warnings (which 
may be suppressed) or serious errors - in fact, warning messages are 
not that simple, most of them are due to misconfigured zones, so it 
might not be a good idea to just ignore them. 


Error messages have to do with serious anomalies, either with the 
packets exchanged with the queried servers (size errors, invalid 
ancounts, nscounts and the like), or others related to the DNS 
information itself (also called "status messages" in the program’s 
documentation): inconsistencies between SOA records as shown by 
different servers for a domain, unexpected address-to-name mappings, 
name servers not responding, not reachable, not running or not 
existing at all, and so on. 


Host performs all its querying on-line, i.e., it only works with data 
received from name servers, which means you have to query a name 
server more than once if you want to get different kinds of reports 
on some particular piece of data. You can always arrange arguments 
in such a way that you get all information you want by running it 
once, but if you forget something or for any reason have to run it 
again, this means extra zone transfers, extra load on name servers, 
extra DNS traffic. 
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Host is an excellent tool, if used carefully. Like most other 
querying programs it may generate lots of traffic, just by issuing a 
simple command. Apart from that, its resolver simulation and debug 
capabilities make it useful to find many common and some not so 
common DNS configuration errors, as well as generate useful reports 
and statistics about the DNS tree. As an example, RIPE (Reseaux IP 
Europeens) NCC uses it to generate a monthly european hostcount, 
giving an overview of the Internet usage evolution in Europe. Along 
with these counts, error reports are generated, one per country, and 


= 


the whole information is made available in the RIPE archive. 


2.2. Dnswalk 


Dnswalk is a DNS debugger written in Perl by David Barr, from 
Pennsylvania State University. You'll find the latest version at 
ftp://ftp.pop.psu.edu/pub/src/dnswalk. With the software comes a 
small document where the author points some useful advice so it may 
be worth reading it. 


The program checks domain configurations stored locally, with data 
arranged hierarchically in directories, resembling the DNS tree 
organization of domains. To set up this information dnswalk may 
first perform zone transfers from authoritative name servers. You can 
have a recursive transfer of a domain and its sub-domains, though you 
should be careful when doing this, as it may generate a great amount 
of traffic. If the data is already present, dnswalk may skip these 
transfers, provided that it is up to date. 


Dnswalk looks for inconsistencies in resource records, such as MX and 
aliases pointing to aliases or to unknown hosts, incoherent PTR, A 
and CNAME records, invalid characters in names, missing trailing 
dots, unnecessary glue information, and so on. It also does some 
checking on authority information, namely lame delegations and 
domains with only one name server. It is easy to use, you only have 
to specify the domain to analyze and some optional parameters and the 
program does the rest. Only one domain (and its sub-domains, if 
that’s the case) can be checked at a time, though. 


While in the process of checking data, dnswalk uses dig and resolver 
routines (gethostbyXXxXxX from the Perl library) a lot, to get such 
data as authority information from the servers of the analyzed 
domains, names from IP addresses so as to verify the existence of PTR 
records, aliases and so on. So, besides the zone transfers you may 
count on some more extra traffic (maybe not negligible if you are 
debugging a relatively large amount of data and care about query 
retries and timeouts), just by running the program. 
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2.3. Lamers 


A lame delegation is a serious error in DNS configurations, yet a 
(too) common one. It happens when a name server is listed in the NS 
records for some domain and in fact it is not a server for that 
domain. Queries are thus sent to the wrong servers, who don’t know 
nothing (at least not as expected) about the queried domain. 
Furthermore, sometimes these hosts (if they exist!) don’t even run 
name servers. As a result, queries are timed out and resent, only to 
fail, thus creating (more) unnecessary traffic. 


It’s easy to create a lame delegation: the most common case happens 
when an administrator changes the NS list for his domain, dropping 
one or more servers from that list, without informing his parent 
domain administration, who delegated him authority over the domain. 
From now on the parent name server announces one or more servers for 
the domain, which will receive queries for something they don’t know 
about. (On the other hand, servers may be added to the list without 
the parent’s servers knowing, thus hiding valuable information from 
them - this is not a lame delegation, but shouldn’t happen either.) 
Other examples are the inclusion of a name in an NS list without 
telling the administrator of that host, or when a server suddenly 
stops providing name service for a domain. 


To detect and warn DNS administrators all over the world about this 
kind of problem, Bryan Beecher from University of Michigan wrote 
lamers, a program to analyze named (the well-known BIND name server) 
logging information [2]. To produce useful logs, named was applied a 
patch to detect and log lame delegations (this patch was originally 
written by Don Lewis from Silicon Systems and is now part of the 
latest release of BIND thanks to Bryan Beecher, so it is expected to 
be widely available in the near future). Lamers is a small shell 
script that simply scans these logs and reports the lame delegations 
found. This reporting is done by sending mail to the hostmasters of 
the affected domains, as stated in the SOA record for each of them. 
If this is not possible, the message is sent to the affected name 
servers’ postmasters instead. Manual processing is needed in case of 
bounces, caused by careless setup of those records or invalid 
postmaster addresses. A report of the errors found by the U-M 
servers is also posted twice a month on the USENET newsgroup 
comp.protocols.tcp-ip.domains. 


If you ever receive such a report, you should study it carefully in 
order to find and correct problems in your domain, or see if your 
servers are being affected by the spreading of erroneous information. 
Better yet, lamers could be run on your servers to detect more lame 
delegations (U-M can’t see them all!). Also, if you receive mail 
reporting a lame delegation affecting your domain or some of your 
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hosts, please don’t just ignore it or flame the senders. They're 
really trying to help! 


You can get lamers from ftp://terminator.cc.umich.edu/dns/lame- 
delegations. 


2.4. DOC 


Authority information is one of the most significant parts of the DNS 
data, as the whole mechanism depends on it to correctly traverse the 
domain tree. Incorrect authority information leads to problems such 
as lame delegations or even, in extreme cases, the inaccessibility of 
a domain. Take the case where the information given about all its 
name servers is incorrect: being unable to contact the real servers 
you may end up being unable to reach anything inside that domain. 
This may be exaggerated, but if you’re on the DNS business long 
enough you’ve probably have seen some enlightened examples of this 
scenario. 


To look for this kind of problems Paul Mockapetris and Steve Hotz, 
from the Information Sciences Institute, wrote a C-shell script 
called DOC (Domain Obscenity Control), an automated domain testing 
tool that uses dig to query the appropriate name servers about 
authority for a domain and analyzes the responses. 


Doc limits its analysis to authority data since the authors 
anticipated that people would complain about such things as invasion 
of privacy. Also, at the time it was written most domains were so 
messy that they thought there wouldn’t be much point in checking 
anything deeper until the basic problems weren’t fixed. 


Only one domain is analyzed each time: the program checks if all the 
servers for the parent domain agree about the delegation information 
for the domain. DOC then picks a list of name servers for the domain 
(obtained from one of the parent’s servers) and starts checking on 
their information, querying each of them: looks for the SOA record, 
checks if the response is authoritative, compares the various records 
retrieved, gets each one’s list of NS, compares the lists (both among 
these servers and the parent’s), and for those servers inside the 
domain the program looks for PTR records for them. 


Due to several factors, DOC seems to have frozen since its first 
public release, back in 1990. Within the distribution there is an 
RFC draft about automated domain testing, which was never published. 
Nevertheless, it may provide useful reading. The software can be 
fetched from ftp://ftp.uu.net/networking/ip/dns/doc.2.0.tar.2Z. 
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20044 DDT 


DDT (Domain Debug Tools) is a package of programs to scan DNS 
information for error detection, developed originally by Jorge Frazao 
from PUUG - Portuguese UNIX Users Group and later rewritten by the 
author, at the time at the Faculty of Sciences of University of 
Lisbon. Each program is specialized in a given set of anomalies: you 
have a checker for authority information, another for glue data, mail 
exchangers, reverse-mappings and miscellaneous errors found in all 
kinds of resource records. As a whole, they do a rather extensive 
checking on DNS configurations. 


These tools work on cached DNS data, i.e., data stored locally after 
performing zone transfers (presently done by a slightly modified 
version of BIND’s named-xfer, called ddt-xfer, which allows recursive 
transfers) from the appropriate servers, rather than querying name 
servers on-line each time they run. This option was taken for 
several reasons [3]: (1) efficiency, since it reads data from disk, 
avoiding network transit delays, (2) reduced network traffic, data 
has to be fetched only once and then run the programs over it as many 
times as you wish and (3) accessibility - in countries with limited 
Internet access, as was the case in Portugal by the time DDT was in 
its first stages, this may be the only practical way to use the 
tools. 


Point (2) above deserves some special considerations: first, it is 
not entirely true that there aren’t additional queries while 
processing the information, one of the tools, the authority checker, 
queries (via dig) each domain’s purported name servers in order to 
test the consistency of the authority information they provide about 
the domain. Second, it may be argued that when the actual tests are 
done the information used may be out of date. While this is true, 
you should note that this is the DNS nature, if you obtain some piece 
of information you can’t be sure that one second later it is still 
valid. Furthermore, if your source was not the primary for the 
domain then you can’t even be sure of the validity in the exact 
moment you got it in the first place. But experience shows that if 
you see an error, it is likely to be there in the next version of the 
domain information (and if it isn’t, nothing was lost by having 
detected it in the past). On the other side, of course there’s 
little point in checking one month old data... 


The list of errors looked for includes lame delegations, version 
number mismatches between servers (this may be a transient problem), 
non-existing servers, domains with only one server, unnecessary glue 
information, MX records pointing to hosts not in the analyzed domain 
(may not be an error, it’s just to point possibly strange or 
expensive mail-routing policies), MX records pointing to aliases, A 
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records without the respective PTR and vice-versa, missing trailing 
dots, hostnames with no data (A or CNAME records), aliases pointing 
to aliases, and some more. Given the specialized nature of each 
tool, it is possible to look for a well defined set of errors, 
instead of having the data analyzed in all possible ways. 


Except for ddt-xfer, all the programs are written in Perl. A new 
release may come into existence in a near future, after a thorough 
review of the methods used, the set of errors checked for and some 
bug fixing (in particular, a Perl version of ddt-xfer is expected). 
In the mean time, the latest version is available from 
ftp://ns.dns.pt/pub/dns/ddt-2.0.1.tar.gz. 


2.6. The Checker Project 


The problem of the huge amount of DNS traffic over the Internet is 
getting researchers close attention for quite some time, mainly 
because most of it is unnecessary. Observations have shown that DNS 
consumes something like twenty times more bandwidth than it should 
[4]. Some causes for this undoubtedly catastrophic scenario lie on 
deficient resolver and name server implementations spread all over 
the world, from personal to super-computers, running all sorts of 
operating systems. 


While the panacea is yet to be found (claims are made that the latest 
official version of BIND is a great step forward [5]), work has been 
done in order to identify sources of anomalies, as a first approach 
in the search for a solution. The Checker Project is one such 
effort, developed at the University of Southern California [6]. It 
consists of a set of C code patched into BIND’s named, for monitoring 
server activity, building a database with the history of that 
operation (queries and responses). It is then possible to generate 
reports from the database summarizing activity and identifying 
behavioral patterns from client requests, looking for anomalies. The 
named code alteration is small and simple unless you want do have PEC 
checking enabled (see below). You may find sources and documentation 
at ftp://catarina.usc.edu/pub/checker. 


Checker only does this kind of collection and reporting, it does not 
try to enforce any rules on the administrators of the defective sites 
by any means whatsoever. Authors hope that the simple exhibition of 
the evidences is a reason strong enough for those administrators to 
have their problems fixed. 


An interesting feature is PEC (proactive error checking): the server 
pretends to be unresponsive for some queries by randomly choosing 

some name and start refusing replies for queries on that name during 
a pre-determined period. Those queries are recorded, though, to try 
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to reason about the retry and timeout schemes used by name servers 
and resolvers. It is expected that properly implemented clients will 
choose another name server to query, while defective ones will keep 
on trying with the same server. This feature seems to be still under 
testing as it is not completely clear yet how to interpret the 
results. A PEC-only error checker is available from USC that is much 
simpler than the full error checker. It examines another name server 
client every 30 minutes to see if this client causes excessive load. 


Presently Checker has been running on a secondary for the US domain 
for more than a year with little trouble. Authors feel confident it 
should run on any BSD platform (at least SunOS) without problems, and 
is planned to be included as part of the BIND name server. 


Checker is part of a research project lead by Peter Danzig from USC, 
aimed to implement probabilistic error checking mechanisms like PEC 
on distributed systems [7]. DNS is one such system and it was chosen 
as the platform for testing the validity of these techniques over the 
NSFnet. It is hoped to achieve enough knowledge to provide means to 
improve performance and reliability of distributed systems. 

Anomalies like undetected server failures, query loops, bad 
retransmission backoff algorithms, misconfigurations and resubmission 
of requests after negative replies are some of the targets for these 
checkers to detect. 


2.7. Others 


All the tools described above are the result of systematic work on 
the issue of DNS debugging, some of them included in research 
projects. For the sake of completeness several other programs are 
mentioned here. These, though just as serious, seem to have been 
developed in a somewhat ad-hoc fashion, without an implicit intention 
of being used outside the environments where they were born. This 
impression is, of course, arguable, nevertheless there was no 
necessity of dedicating an entire section to any of them. This 
doesn’t mean they are not valuable contributions, in some cases they 
may be just what you are looking for, without having to install a 
complete package to do some testings on your domain. 


The reference taken was the contrib directory in the latest BIND 
distribution (where some of the above programs can also be found). 
There you will find tools for creating your DNS configuration files 
and NIS maps from /etc/hosts and vice-versa or generate PTR from A 
records (these things may be important as a means of avoiding common 
typing errors and inconsistencies between those tables), syntax 
checkers for zone files, programs for querying and monitoring name 
servers, all the small programs presented in [8], and more. It is 
worth spending some time looking at them, maybe you’1ll find that 
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program you were planning to write yourself. The latest public 
version of BIND can be found at 
ftp://gatekeeper.dec.com/pub/misc/vixie/4.9.2-940221.tar.gz. As of 
this writing BIND-4.9.3 is in its final beta stages and a public 
release is expected soon, also at gatekeeper.dec.com. 


You may also want to consider using a version control system like 
SCCS or RCS to maintain your configuration files consistent through 
updates, or use tools like M4 macros to generate those files. As 
stated above, it’s important to avoid human-generated errors, 
creating problems that are difficult to track down, since they’re 
often hidden behind some mistyped name. Errors like this may end up 
in many queries for a non-existing name, just to mention the less 
serious kind. See [9] for a description of the most common errors 
made while configuring domains. 


3. Why look after DNS? 


Several pieces of software were presented to help people administer 
and debug their name services. They exhibit many differences in 
their way of doing things, scope and requirements and it may be 
difficult just to choose one of them to work with. For one thing, 
people’s expectations from these tools vary according to their kind 
of involvement with DNS. If you are responsible for a big domain, 
e.g., a top-level one or a big institution with many hosts and sub- 
domains, you probably want to see how well is the tree below your 
node organized, since the consequences of errors tend to propagate 
upwards, thus affecting your own domain and servers. For that you 
need some program that recursively descends the domain tree and 
analyzes each domain per se and the interdependencies between them 
all. You will have to consider how deep you want your analysis to 
be, the effects it will have on the network infrastructure, i.e., 
will it generate traffic only inside a campus network, no matter how 
big it is, or will it be spread over, say, a whole country (of 
course, your kind of connectivity plays an important role here). 


You may simply want to perform some sanity checks on your own domain, 
without any further concerns. Or you may want to participate in some 
kind of global effort to monitor name server traffic, either for 
research purposes or just to point out the "trouble-queries" that 
flow around. 


Whatever your interest may be, you can almost surely find a tool to 
suit it. Eliminating problems like those described in this document 
is a major contribution for the efficiency of an important piece of 
the Internet mechanism. Just to have an idea of this importance, 
think of all the applications that depend on it, not just to get 
addresses out of names. Many systems rely on DNS to store, retrieve 
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and spread the information they need: Internet electronic mail was 
already mentioned (see [10] for details) and work is in progress to 
integrate X.400 operations with DNS [11]; others include "remote 
printing" services [12], distributed file systems and network routing 
purposes, among others. These features may be accomplished by some 
standard, well-known resource records [13], or by new, experimental 
ones [14, 15]. Even if some of them won’t succeed, one may well 
expect some more load on the DNS burden. 


The ubiquitous DNS thus deserves a great deal of attention, perhaps 
much more than it generally has. One may say that it is a victim of 
its own success: if a user triggers an excessive amount of queries 
only to have one request satisfied, he won’t worry about it (in fact, 
he won’t notice it), won’t complain to his system administrator, and 
things will just go on like this. Of course, DNS was designed to 
resist and provide its services despite all these anomalies. But by 
doing so it is frequently forgotten, as long as people can Telnet or 
ftp. As DNS will be given new responsibilities, as pointed in the 
above paragraph, the problems described in this text will grow more 
serious and new ones may appear (notably security ones [16], witha 
lot of work being presently in progress addressing security in DNS), 
if nothing is done to purge them. 
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Security Considerations 


Security issues are not discussed in this memo (although security is 
briefly mentioned at the end of section 3). 
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